Skip to content

Add end-of-file-fixer to pre-commit#106065

Merged
hugovk merged 4 commits intopython:mainfrom
hugovk:patchcheck-end-of-file-fixer
Jun 25, 2023
Merged

Add end-of-file-fixer to pre-commit#106065
hugovk merged 4 commits intopython:mainfrom
hugovk:patchcheck-end-of-file-fixer

Conversation

@hugovk
Copy link
Member

@hugovk hugovk commented Jun 24, 2023

Follow on from #104275.

This adds a check that is part of make patchcheck's normalize_whitespace(python_files) to pre-commit, which showed up in PR #105984:

Getting base branch for PR ... origin/main
Getting the list of files that have been added/changed ... 3 files
Fixing Python file whitespace ... 1 file:
  Lib/test/test_webbrowser.py
Fixing C file whitespace ... 0 files
Fixing docs whitespace ... 0 files
Please fix the 1 file(s) with whitespace issues
(on UNIX you can run `make patchcheck` to make the fixes)
##[error]Bash exited with code '1'.

https://dev.azure.com/Python/cpython/_build/results?buildId=130810&view=logs&j=256d7e09-002a-52d7-8661-29ee3960640e&t=3d7276d3-4e8d-5309-55ad-fb0b172d9925

Adding here will make the failure more visible, and show the actual change needed.

I'll push another temporary commit to demonstrate.

@hugovk
Copy link
Member Author

hugovk commented Jun 24, 2023

Temporary commit not needed, as there's a file that needs fixing:

check yaml...............................................................Passed
fix end of files.........................................................Failed
- hook id: end-of-file-fixer
- exit code: 1
- files were modified by this hook

Fixing Lib/test/coding20731.py

trim trailing whitespace.................................................Passed
Sphinx lint..............................................................Passed
pre-commit hook(s) made changes.
If you are seeing this message in CI, reproduce locally with: `pre-commit run --all-files`.
To run `pre-commit` as part of git workflow, use `pre-commit install`.
All changes made by hooks:
diff --git a/Lib/test/coding20731.py b/Lib/test/coding20731.py
index b0e227a..7b61917 [100](https://github.com/python/cpython/actions/runs/5366030779/jobs/9735224499?pr=106065#step:4:105)644
--- a/Lib/test/coding20731.py
+++ b/Lib/test/coding20731.py
@@ -1,4 +1 @@
 #coding:latin1
-
-
-
Error: Process completed with exit code 1.

https://github.com/python/cpython/actions/runs/5366030779/jobs/9735224499?pr=106065

@hugovk
Copy link
Member Author

hugovk commented Jun 24, 2023

Like trailing-whitespace, should we run on C and RST in addition to Python files?

diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index e21feee3ed..78627f9839 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -4,7 +4,7 @@ repos:
     hooks:
       - id: check-yaml
       - id: end-of-file-fixer
-        types_or: [python]
+        types_or: [c, python, rst]
       - id: trailing-whitespace
         types_or: [c, python, rst]

If so, it would do an initial tidy-up of these files:

Files 
#       modified:  [1] .pre-commit-config.yaml
#       modified:  [2] Doc/c-api/allocation.rst
#       modified:  [3] Doc/c-api/frame.rst
#       modified:  [4] Doc/c-api/gcsupport.rst
#       modified:  [5] Doc/c-api/init.rst
#       modified:  [6] Doc/c-api/intro.rst
#       modified:  [7] Doc/c-api/long.rst
#       modified:  [8] Doc/c-api/marshal.rst
#       modified:  [9] Doc/copyright.rst
#       modified: [10] Doc/faq/installed.rst
#       modified: [11] Doc/howto/index.rst
#       modified: [12] Doc/howto/instrumentation.rst
#       modified: [13] Doc/howto/ipaddress.rst
#       modified: [14] Doc/howto/sockets.rst
#       modified: [15] Doc/howto/urllib2.rst
#       modified: [16] Doc/library/_thread.rst
#       modified: [17] Doc/library/array.rst
#       modified: [18] Doc/library/base64.rst
#       modified: [19] Doc/library/binary.rst
#       modified: [20] Doc/library/copy.rst
#       modified: [21] Doc/library/curses.ascii.rst
#       modified: [22] Doc/library/curses.panel.rst
#       modified: [23] Doc/library/dbm.rst
#       modified: [24] Doc/library/email.charset.rst
#       modified: [25] Doc/library/email.encoders.rst
#       modified: [26] Doc/library/email.header.rst
#       modified: [27] Doc/library/filecmp.rst
#       modified: [28] Doc/library/getopt.rst
#       modified: [29] Doc/library/gzip.rst
#       modified: [30] Doc/library/heapq.rst
#       modified: [31] Doc/library/imaplib.rst
#       modified: [32] Doc/library/logging.handlers.rst
#       modified: [33] Doc/library/mailbox.rst
#       modified: [34] Doc/library/marshal.rst
#       modified: [35] Doc/library/modulefinder.rst
#       modified: [36] Doc/library/pyexpat.rst
#       modified: [37] Doc/library/rlcompleter.rst
#       modified: [38] Doc/library/shelve.rst
#       modified: [39] Doc/library/site.rst
#       modified: [40] Doc/library/socketserver.rst
#       modified: [41] Doc/library/stringprep.rst
#       modified: [42] Doc/library/termios.rst
#       modified: [43] Doc/library/text.rst
#       modified: [44] Doc/library/trace.rst
#       modified: [45] Doc/library/tty.rst
#       modified: [46] Doc/library/urllib.error.rst
#       modified: [47] Doc/library/uuid.rst
#       modified: [48] Doc/library/winreg.rst
#       modified: [49] Doc/library/winsound.rst
#       modified: [50] Doc/library/wsgiref.rst
#       modified: [51] Doc/library/xml.dom.rst
#       modified: [52] Doc/library/xml.sax.reader.rst
#       modified: [53] Doc/library/xml.sax.rst
#       modified: [54] Doc/library/xml.sax.utils.rst
#       modified: [55] Doc/library/zipapp.rst
#       modified: [56] Doc/reference/introduction.rst
#       modified: [57] Doc/tutorial/appetite.rst
#       modified: [58] Doc/tutorial/stdlib2.rst
#       modified: [59] Doc/tutorial/whatnow.rst
#       modified: [60] Doc/using/mac.rst
#       modified: [61] Doc/whatsnew/2.0.rst
#       modified: [62] Doc/whatsnew/2.1.rst
#       modified: [63] Doc/whatsnew/2.2.rst
#       modified: [64] Doc/whatsnew/2.4.rst
#       modified: [65] Doc/whatsnew/2.5.rst
#       modified: [66] Doc/whatsnew/2.6.rst
#       modified: [67] Doc/whatsnew/3.5.rst
#       modified: [68] Include/cpython/longobject.h
#       modified: [69] Include/internal/pycore_ast_state.h
#       modified: [70] Include/internal/pycore_blocks_output_buffer.h
#       modified: [71] Include/internal/pycore_weakref.h
#       modified: [72] Include/iterobject.h
#       modified: [73] Include/py_curses.h
#       modified: [74] Include/pyexpat.h
#       modified: [75] Include/warnings.h
#       modified: [76] Mac/BuildScript/README.rst
#       modified: [77] Misc/NEWS.d/next/Core and Builtins/2023-06-11-09-14-30.gh-issue-105331.nlZvoW.rst
#       modified: [78] Misc/coverity_model.c
#       modified: [79] Modules/_blake2/impl/blake2-config.h
#       modified: [80] Modules/_blake2/impl/blake2-impl.h
#       modified: [81] Modules/_blake2/impl/blake2.h
#       modified: [82] Modules/_blake2/impl/blake2b-load-sse2.h
#       modified: [83] Modules/_blake2/impl/blake2b-load-sse41.h
#       modified: [84] Modules/_blake2/impl/blake2b-ref.c
#       modified: [85] Modules/_blake2/impl/blake2b-round.h
#       modified: [86] Modules/_blake2/impl/blake2s-load-sse41.h
#       modified: [87] Modules/_blake2/impl/blake2s-load-xop.h
#       modified: [88] Modules/_blake2/impl/blake2s-ref.c
#       modified: [89] Modules/_blake2/impl/blake2s-round.h
#       modified: [90] Modules/_blake2/impl/blake2s.c
#       modified: [91] Modules/_decimal/docstrings.h
#       modified: [92] Modules/_decimal/libmpdec/examples/compare.c
#       modified: [93] Modules/_decimal/libmpdec/examples/div.c
#       modified: [94] Modules/_decimal/libmpdec/examples/divmod.c
#       modified: [95] Modules/_decimal/libmpdec/examples/multiply.c
#       modified: [96] Modules/_decimal/libmpdec/examples/pow.c
#       modified: [97] Modules/_decimal/libmpdec/examples/powmod.c
#       modified: [98] Modules/_decimal/libmpdec/examples/shift.c
#       modified: [99] Modules/_decimal/libmpdec/examples/sqrt.c
#       modified: [100] Modules/_hacl/Hacl_Hash_MD5.c
#       modified: [101] Modules/_hacl/Hacl_Hash_SHA1.c
#       modified: [102] Modules/_hacl/Hacl_Hash_SHA2.c
#       modified: [103] Modules/_hacl/Hacl_Hash_SHA3.c
#       modified: [104] Modules/_ssl_data_111.h
#       modified: [105] Modules/_ssl_data_300.h
#       modified: [106] Modules/_ssl_data_31.h
#       modified: [107] Modules/_testbuffer.c
#       modified: [108] Modules/_testimportmultiple.c
#       modified: [109] Modules/cjkcodecs/emu_jisx0213_2000.h
#       modified: [110] Modules/cjkcodecs/mappings_hk.h
#       modified: [111] Modules/cjkcodecs/mappings_jp.h
#       modified: [112] Modules/cjkcodecs/mappings_kr.h
#       modified: [113] Modules/cjkcodecs/mappings_tw.h
#       modified: [114] Modules/expat/pyexpatns.h
#       modified: [115] Modules/getpath.c
#       modified: [116] Modules/hashlib.h
#       modified: [117] Modules/unicodedata_db.h
#       modified: [118] Objects/bytesobject.c
#       modified: [119] Objects/capsule.c
#       modified: [120] Objects/exceptions.c
#       modified: [121] Objects/stringlib/fastsearch.h
#       modified: [122] Objects/stringlib/find_max_char.h
#       modified: [123] Objects/stringlib/partition.h
#       modified: [124] Objects/stringlib/split.h
#       modified: [125] Objects/stringlib/ucs4lib.h
#       modified: [126] Objects/unicodectype.c
#       modified: [127] Objects/unicodetype_db.h
#       modified: [128] PC/frozen_dllmain.c
#       modified: [129] Programs/_bootstrap_python.c
#       modified: [130] Programs/_freeze_module.c
#       modified: [131] Python/Python-ast.c
#       modified: [132] Python/ceval_gil.c
#       modified: [133] Python/getopt.c
#       modified: [134] Python/pyctype.c
#       modified: [135] Python/suggestions.c
#       modified: [136] Python/traceback.c
#       modified: [137] Tools/msi/bundle/bootstrap/pch.h

If not, we could simplify a bit:

diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index e21feee3ed..1197ef745e 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -4,7 +4,7 @@ repos:
     hooks:
       - id: check-yaml
       - id: end-of-file-fixer
-        types_or: [python]
+        types: [python]
       - id: trailing-whitespace
         types_or: [c, python, rst]

@hugovk hugovk marked this pull request as ready for review June 24, 2023 20:26
@hugovk hugovk requested a review from AlexWaygood as a code owner June 24, 2023 20:26
AlexWaygood
AlexWaygood reviewed Jun 25, 2023
View reviewed changes
@AlexWaygood
Copy link
Member

AlexWaygood commented Jun 25, 2023 •
edited
Loading

Like trailing-whitespace, should we run on C and RST in addition to Python files?

Let's do the hopefully-uncontroversial bit (adding it for .py files) first, and we can consider other file extensions separately in followup PRs, since they'd lead to much bigger diffs

@hugovk hugovk added the needs backport to 3.12 only security fixes label Jun 25, 2023
AlexWaygood
AlexWaygood approved these changes Jun 25, 2023
View reviewed changes
Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@hugovk hugovk merged commit 8c24a83 into python:main Jun 25, 2023
@hugovk hugovk deleted the patchcheck-end-of-file-fixer branch June 25, 2023 10:44
@miss-islington
Copy link
Contributor

miss-islington commented Jun 25, 2023

Thanks @hugovk for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

@bedevere-bot
Copy link

bedevere-bot commented Jun 25, 2023

GH-106080 is a backport of this pull request to the 3.12 branch.

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jun 25, 2023
@hugovk @miss-islington
Add end-of-file-fixer to pre-commit (pythonGH-106065)
dcb16cf 
(cherry picked from commit 8c24a83)

Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
@bedevere-bot bedevere-bot removed the needs backport to 3.12 only security fixes label Jun 25, 2023
AlexWaygood pushed a commit that referenced this pull request Jun 25, 2023
@miss-islington @hugovk
[3.12] Add end-of-file-fixer to pre-commit (GH-106065) (#106080)
f955ed9 
Add end-of-file-fixer to pre-commit (GH-106065)
(cherry picked from commit 8c24a83)

Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@AlexWaygood AlexWaygood AlexWaygood approved these changes

Assignees

No one assigned

Labels

skip issue skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@hugovk @AlexWaygood @miss-islington @bedevere-bot